Topic Shift Analysis & Validation
Overview
Now that we have labels for out topic we can plot the topic shift and review statistics that might help us understand trends across the different models.
These metrics describe the behavior and nature of each topic over time.
Topic Volatility
How “spiky” or “stable” is a topic’s prominence over time?
For each topic, calculate the standard deviation or coefficient of variation of its average gamma scores across all speeches.
Interpretation: A high volatility score indicates an “event-driven” topic that appears intensely and then fades (e.g., war, pandemic). A low score suggests a stable, persistent “background” topic that is always present.
Topic Persistence
Does a topic remain relevant long after its peak, or does it disappear completely? For each topic, calculate a “persistence score” by dividing its average gamma in the most recent year by its all-time peak gamma.
Interpretation: A score close to 1 means the topic is still highly relevant today, even if its peak was long ago. A score close to 0 means the topic has largely faded from the discourse.
Topic Exclusivity
How unique are the words that define a topic? For each topic, count how many of its top 15 words are not in the top 15 words of any other topic in the same model.
Interpretation: High exclusivity suggests a very distinct and well-defined theme. Low exclusivity suggests a more general or foundational topic whose vocabulary is shared across many different policy areas.
4 Topics
We use the 4 topic model for more defined and interpretable topics.| Topic | Peak Year | Volatility | Persistence | Exclusivity | |
|---|---|---|---|---|---|
| International Affairs & Defense | 1878 | 0.45 | 0 | 17 | |
| Parliamentary Procedure & Administration | 1946 | 0.37 | 0 | 12 | |
| Economic Development & Growth | 1980 | 0.32 | 0 | 9 | |
| Social Policy & Community Welfare | 2013 | 0.34 | 1 | 16 |
The plot is striking. The topics are well defined and a clear pattern of a synchronized raising and declining prominence (stable Volatility). The topic labels seem to fit well with the eras each topic represents.
8 Topics
The 8 topics model is more temporal. Even though there more topics they are evenly exclusive.
| Topic | Peak Year | Volatility | Persistence | Exclusivity | |
|---|---|---|---|---|---|
| Federal-Provincial Coordination | 1910 | 0.45 | 0.00 | 12 | |
| Parliamentary Business & Trade | 1931 | 0.30 | 0.00 | 9 | |
| War & International Relations | 1944 | 0.28 | 0.00 | 10 | |
| Jobs & Economic Policy | 1968 | 0.20 | 0.00 | 5 | |
| Infrastructure & Regional Development | 1976 | 0.21 | 0.00 | 8 | |
| Economic Development Programs | 1983 | 0.23 | 0.40 | 6 | |
| Indigenous Affairs & Climate | 2013 | 0.23 | 0.40 | 7 | |
| Social Programs & Healthcare | 2020 | 0.24 | 0.21 | 8 |
We now see topic that declined in prominence and later reemerge, like “War & International Relations”. We see periods where topics peek with less prominence. Topics in the early and late years, appear to have strong prominence.
Validation against Major Events
To validate the topics prominence we compare them against major events and governments.
Major Canadian Historical Events
The table below lists partial but uncontroversial, major Canadian historical events marked in the plots below for.
| Date | Event |
|---|---|
| 1867 | Confederation: The Dominion of Canada is formed. |
| 1885 | Completion of the Canadian Pacific Railway. |
| 1914 | Canada enters World War I. |
| 1918 | World War I ends. |
| 1929 | Great Depression begins. |
| 1939 | Canada enters World War II. |
| 1945 | World War II ends. |
| 1959 | Suez Crisis |
| 1973 | Oil Embargo |
| 1982 | Canada Act 1982: Patriation of the Constitution. |
| 1995 | Widespread adoption of the internet. |
| 2001 | Canada joins the Afghanistan War. |
| 2020 | COVID-19 pandemic: WHO declares a global pandemic. |
Governments
We can also compare the topics prominence against the different governments and see if they “fit”.
Conclusion
This topic modeling analysis of Canadian Speeches from the Throne reveals meaningful patterns in political discourse that align closely with historical events and governmental transitions. The 4-topic model demonstrates clear temporal succession, with Parliamentary Procedure & Administration dominating the early confederation period, followed by International Affairs & Defense during the world wars, Economic Development & Growth in the post-war boom, and Social Policy & Community Welfare in the modern era. The 8-topic model provides additional granularity, capturing the cyclical nature of themes like “War & International Relations” that resurge during different conflicts, and revealing how contemporary issues like Indigenous Affairs & Climate have gained prominence in recent decades. The validation against major historical events—from Confederation through both World Wars, the Great Depression, and the COVID-19 pandemic—confirms that these computational methods successfully identify genuine shifts in political priorities.
These findings demonstrate that unsupervised machine learning can effectively uncover the evolution of political discourse, providing quantitative evidence for how Canadian political priorities have shifted across nearly 160 years of parliamentary debate. The use of Speeches from the Throne as our corpus was particularly valuable, as these ceremonial addresses represent a standardized format delivered at consistent intervals, creating a reliable yardstick for measuring thematic changes over time. Unlike other parliamentary speeches that vary widely in context and purpose, Throne Speeches serve as an index of governmental priorities, allowing us to track shifts in political focus with greater precision and comparability across different eras and administrations.